Is it easy to launch your own Kotlin Ktor server f...
# server
j
Is it easy to launch your own Kotlin Ktor server from scratch act as proxy to Firebase Admin SDK, easy as in can do with AI agent or generate most of the boilerplate? It doesnt have to be perfect, but good enough to test if its worth it. Just deploy directly to GCP or something. Or something equivalent of that. Can be Apollo GraphQL or whatever.
βœ… 1
m
Not sure what you're trying to do? This all sounds doable. You don't even need GraphQL for that.
Is the main thing that you want most of the code to be generated automatically?
j
Its quite eh weird. Because firebase sdk not support wasm I am doing a new backend earlier than I wanted. I wanted to do that anyway but thought about wait. Ideally want AI write all code and deploy it for me if possible as my backend is a little rusty. The main thing is do as little as possible as cheap as possible fast. Its an internal project so doesnt need to be perfect.
Ideally want to give my firebase project access to AI and rewrite my existing firebase firestore code into backend api I can consume. I will consume from desktop, web, Android and iOS.
As its KMP the ideal case would be if backend can share Kotlin code, like data objects and kotlinx serialization.
s
Just deploy directly to GCP or something.
To easily deploy Ktor to GCP use the Ktor Gradle Plugin. It can upload directly to Google's Container Registry, which you can then easily deploy into Google Cloud. An example. The official Ktor samples has an example of a reverse proxy build in Ktor, so you could clone and adjust it a bit to your needs. https://github.com/ktorio/ktor-samples/tree/main/reverse-proxy. By combining those two it should be fairly easy even without AI ☺️ Let me know if you have any issues or questions, or checkout the #C0A974TJ9 channel for Ktor specific issues.
πŸš€ 2
j
Sweet! Also heard about Google App Engine to easy deploy, is it easy? When I say easy like one click deploy without hassle. Remember years ago it was painful deploy and config. I bet most annoying part will be config security protection for api and token keys. Like fetch firebase service account in gcp to communicate with firebase admin SDK. Thanks for the pointers, really appreciate it! πŸ₯³ I ofc dont mind learn and code, its just time is of essence. I am normally coding Android apps. It will be interesting use case for me. Hopefully I can share my existing Ktor client setup with Ktor server somewhat.
m
Google App Engine
I would forget about App Engine. I wouldn't be surprised it'd be deprecated in favor of Cloud Run soon
☝️ 2
πŸ‘€ 1
s
For new Google Cloud users, we recommend using Cloud Run as the preferred alternative over App Engine.
Yes, I was just going to say. Cloud Run is the new kid on the block, so I would go with that. It's almost 1-click to deploy if your app is already in the Google Container Registry.
m
I share the general sentiment though. Writing the code is the "easy" part. After writing a couple of servers, I find deploying to be a bit confusing. Navigating the GCP marketing terms, dealing with secrets, etc... takes some effort
βž• 2
I wish we could "just" share terraform config but it's not that easy
πŸ˜… 2
Like I've had "entity workload federation" on my todo list for a while now
πŸ‘€ 1
j
Thanks, its exactly these kind of things want to learn if using Cloud Run is the easiest today I do that πŸ™‚ And yeah both deploying mobile apps and backend is pain in the ass. Comply Google and Apple policies on mobile apps, or cloud constraints on backend deployment with insane amount of config forget same day you deployed how you did, if not deploy new servers daily πŸ˜„ Often having Devops or sysops doing this on full time on companies, now trying do all of this myself. Its for a small API and small mobile app however so nothing fancy. Is there any recommendations where storing data as well as deploy secrets, without have to pay insane amount of money for 50 different tools like Vault or something? Also I wish there was like good old web hosts, had 1 click install Wordpress and select what you want and modify it πŸ˜„ Terraform, Github workflows and GCP with all these sys ops stuff, and also Gradle build system itself is complex how deploy with different plugins. Mostly because best practice very in insane speed every week in all platforms, impossible to keep up. There is one like large enterprise way of doing things vs small company doing tiny internal project so to say. In some cases enterprise solutions being to picky in some stuff being paranoid protect everything and do overkill tooling just for the sake of it.
πŸ’― 1
I wish there was like Ktor Docker container, click on it in GCP and then connect to it from Android Studio/IDE and deploy with one button from IDE πŸ˜„
@mbonnin I can really recommend "Workload Identity Federation" btw, used it a lot in earlier places. Also much easier than the alternative actually for most cases, as long as having easy organized GCP setup with permissions, and not something custom πŸ˜„
πŸ‘€ 1
m
@Joel Denke will do! My worst nightmare is my service-account.json leaking and having to pay $100k in cloud fees because someone used it to mine bitcoins 😬
πŸ’€ 2
πŸ˜‚ 1
While we're on this topic, do any of you have an easy way to deploy a new revision of an image? I use the Cloud Run Java SDK to create a new revision where I tweak the revision name but this feels a bit awkward....
Maybe this is the way to go, it's just since I'm a relative noob in docker, etc..., should I use the git sha1 as revision for my images? Or should I rather use tags (which is what I'm currently doing I think with a
latest
tag)? Do you have a favorite workflow there?
BTW, if you're anxious, I strongly discourage reading https://www.reddit.com/r/googlecloud/, the number of horror stories there is daunting...
🀣 2
s
A lot of people are "noob" in this area unless they do it full time like @Joel Denke mentioned.
While we're on this topic, do any of you have an easy way to deploy a new revision of an image? I use the Cloud Run Java SDK to create a new revision where I tweak the revision name but this feels a bit awkward...
Yes, I publish it from CI using Gradle using the Ktor Plugin. Most companies I worked at we used a combination of SHA + semantic versioning of the application. SHA alone should be sufficient but they're hard to read and reason. That in combination with staged rollouts is a pretty nice workflow. @mbonnin I think your
BuildImageTask
can be replaced by the Ktor plugin. It also uses JIB under the hood. I see you're on Ktor 2.x.x, so maybe the plugin didn't exist at that time? πŸ€”
πŸ‘€ 1
πŸ‘ 1
m
I wasn't aware there was a
ktor
plugin πŸ˜…
Ideally, I was expecting such a plugin to be server-agnostic. Just give it jar + runtime classpath and package an image
I looked into the google plugins back in the day but didn't find anything
https://ktor.io/docs/google-app-engine.html#configure-app-engine-plugin Wasn't specifically that page but this is the kind of stuff I got lost in at the time. I would probably recommend removing that page altogether
πŸ’― 1
s
Hmm, that's actually a great point... Here is the repository btw, https://github.com/ktorio/ktor-build-plugins/tree/main/plugin/src/main/kotlin/io/ktor/plugin/features. Briefly checking the code it also seems to supports building images for KMP apps... that could be super useful.
kodee loving 2
Yes.. horrible page. I have it on my agenda...
πŸ™Œ 1
❀️ 1
j
Wow so much cool stuff here, Apollo Execution seems lovely will test that as well, as curious try out using GraphQl for this project πŸ™‚ I am glad I have my gradle convention plugins and all that now, as easy re-use a lot of code from my client apps into server as well. Seems like KMP monorepo is the way to go here for my case. Setting up Cloud Run and all that now. Just not sure when want FatJar, Docker or whatever binary, I just want a deploy to GCP I dont care which one it is. I guess they exist so can publish to maven or something and then deploy it externally. But yeah this is all the noise I dont want, just want basic GCP deploy of Kotlin server, as little code as possible πŸ˜„ Feels like I need to start doing Sys ops and backend API learning again. Hopefully not a lot of boilerplate of configuration, where I only want to proxy Firebase Admin because they dont support Wasm target lol πŸ˜„
❀️ 1
πŸš€ 1
m
You want Jib for container images without docker. Doesn't have to be a fat jar, you can just copy the runtime classpath.
j
At moment just borrowed setup like this for now:
Copy code
Jib.from("openjdk:17-alpine")
            .addLayer(listOf(path), AbsoluteUnixPath.get("/"))
            .addLayer(runtimeClasspath.files.map { it.toPath() }, AbsoluteUnixPath.get("/classpath"))
            .setEntrypoint(
                "java",
                "-cp",
                (runtimeClasspath.files.map { "classpath/${it.name}" } + path.name).joinToString(":"),
                mainClass.get())
            .containerize(containerizer)
Not even sure if that what I want, but testing at the moment πŸ˜„
m
It should be alright πŸ‘ Or you can reuse the ktor Gradle plugin for that part too
j
No idea how to easy setup Google Federated Identity to get easy setup, only care about getting out Google Application credentials auth and the rest automatic safely taken care of, work both in CI pipeline and locally. At the moment I dont care about CI, just creating basic gradle tasks to deploy this πŸ˜„
Really, does Ktor doing this πŸ˜„
Is the equivalent of this using something like this?
Copy code
ktor {
    docker {
        jreVersion.set(JreVersion.JRE_21)
        localImageName.set("sample-docker-image")
        imageTag.set("0.0.1-preview")

        externalRegistry.set(
            DockerImageRegistry.dockerHub(
                appName = provider { "ktor-app" },
                username = providers.environmentVariable("DOCKER_HUB_USERNAME"),
                password = providers.environmentVariable("DOCKER_HUB_PASSWORD")
            )
        )
    }
}
m
I'm not sure about ktor really. The
docker
terminology sounds a bit suspicious there. jib != docker
But maybe it's a simplification
j
What does Jib actually do behind the curtains? πŸ˜„ I dont care what its using, as long as working in GCP πŸ˜„
s
No, native image is GraalVM. I don't think you want that. Please just use the the Ktor Gradle plugin, it also uses Jib. Jib layers your container, or in other words when you redeploy and only your main sources change only your main sources get rebuild into your container. Your dependencies can stay as they are, and build times reduced.
☝️ 1
m
@simon.vergauwen why does ktor say "docker" if it's using Jib?
j
Yeah, but looks like ktor plugin using fatjar, native image or docker?
s
@simon.vergauwen why does ktor say "docker" if it's using Jib?
Jib builds optimized Docker and OCI images for your Java applications without a Docker daemon - and without deep mastery of Docker best-practices. Jib is still docker πŸ˜…
m
@Joel Denke cloud images (for lack of a better word) is just a tar files containing a small linux distribution and your jars.
☝️ 1
j
Also btw really appreciate all this feedback, and hope this thread can help other "newbies" like me πŸ™‚
m
Is OCI docker?
I thought OCI was a superset of docker
j
To much buzz words now πŸ˜„ WHich variant of ktor plugin should I use, cant find like pure sample using Jib layer container approach?
m
I.e. you can build an OCI image that doesn't require or use docker at all
Docker being a company that implements OCI
s
Yes, Docker is a vendor and OCI is a spec
j
If ignoring Docker at moment, should I use like application + fatjar or what combo in Ktor plugin, not following what you recommend πŸ˜„
m
I would replace
docker {}
by
ociImage {}
then to avoid conflating the brand and the technology
πŸ‘ 1
s
m
@Joel Denke you want
docker {}
. Sorry if I'm bringing confusion here by nitpicking the name
ktor using "docker" terminology here is like people using "apollo" for graphql
j
So like:
Copy code
ktor {
    docker {
        localImageName = "ktor-ai-example"
        imageTag = project.version.toString()
        externalRegistry =
            googleContainerRegistry(
                projectName = provider { "Kotlin Conf 2025" },
                appName = provider { project.name },
                username = providers.environmentVariable("GCLOUD_REGISTRY_USERNAME"),
                password = providers.environmentVariable("GCLOUD_REGISTRY_PASSWORD"),
            )
    }
    fatJar {
        allowZip64 = true
        archiveFileName.set(project.name)
    }
}
m
I'd say yes
j
Yeah cool, then I just will figure out how to inject the credentials from GCP in safe way πŸ˜„
m
Please share the code when you have it, I'll update all my repos ^^
j
Yes sure will do πŸ™‚ Trying to grasp the bare minimum setup with ktor server as well. I can share when I know I can run hello World πŸ˜„
❀️ 2
Btw is it good idea run server on KMP, or better run as pure Kotlin library?
❓ 2
s
Not sure I got it right, but you want Kotlin/JVM with the Netty engine. Feel free to ask any other question if you have any, this has been quite valuable for me to improve things on the Ktor side ☺️
j
Its possible to run server as Kotlin multiplatform module, but regular Kotlin JVM maybe preferred? Trying to see if its worth sharing my gradle convention plugin setup I have with other KMP modules πŸ™‚
m
That's something I've been wanting to benchmark for a while. My hunch is JVM gives you the highest throughput but native could be interesting if your load is very uneven
If you're starting/stopping a lot of small instances, K/N or GraalVM will give you faster start time and probably lower memory usage
But one big warm JVM instance will probably outperform that it terms of raw throughput
So it's a tradeoff. JVM (Netty) is definitely easier to start with because of all the existing code and documentation
s
native could be interesting if your load is very uneven
I think GraalVM Netty would outperform Kotlin/Native with CIO due to Netty being more efficient but that'd be a super interesting test actually πŸ˜„
πŸ’― 1
j
Copy code
[bundles]
ktor = [
    "ktor-json",
    "ktor-logging",
    "ktor-content-negotiation"
]
ktor-server = [
    "ktor-server-sessions",
    "ktor-server-core",
    "ktor-server-cors",
    "ktor-server-netty"
]

google-cloud = [
    "google-cloud-datastore",
    "google-cloud-run",
    "google-cloud-storage",
    "google-cloud-tools-jib-core"
]
This kind of setup I wish was predefinied somewhere for stuff, not only BOM but also predefinied constants of all combinations. I did see some libs doing this, like Koin πŸ˜„
s
j
In most cases I just want like libs.ktor.server and dont care about which server its using, just want the most recent / recommended one most people using πŸ˜„
Oh wow damn thanks, missed out that this existed:
Copy code
versionCatalogs {
        create("ktorLibs") {
            from("io.ktor:ktor-version-catalog:3.2.0")
        }
    }
Cant the version for that also be applied from my own libs TOML file? πŸ˜„
Copy code
create("ktorLibs") {
            from(libs.ktorVersionCatalor)
        }
Copy code
dependencies {
    implementation(libs.bundles.ktor.server)
}
Ah well I go with this for now, and will see how can improve later on, with nesting version catalogs with custom bundle point to another version catalog πŸ˜„
@mbonnin This is what I do now:
Copy code
import io.ktor.plugin.features.DockerImageRegistry

plugins {
    alias(libs.plugins.ktor)
    kotlin("jvm")
}

application.mainClass = "sample.ApplicationKt"

ktor {
    docker {
        jreVersion = JavaVersion.VERSION_17
        localImageName = "sample-docker-image"
        imageTag =project.version.toString()
        externalRegistry = DockerImageRegistry.googleContainerRegistry(
            projectName = provider { "Project" },
            appName = provider { project.name },
            username = providers.environmentVariable("GCLOUD_REGISTRY_USERNAME"),
            password = providers.environmentVariable("GCLOUD_REGISTRY_PASSWORD"),
        )
        fatJar {
            allowZip64 = true
            archiveFileName.set(project.name)
        }
    }
}

dependencies {
    implementation(libs.bundles.ktor.server)
}
m
> Or you can reuse the ktor Gradle plugin for that part too I take that back. I've been looking at the ktor Gradle plugin and I'll continue copy/pasting my AndroidMakers backend code for the time being because: β€’ the ktor plugin is not configuration cache compatible β€’ it looks like it requires
docker
even when running Jib (which kind of makes my naming point above numb) Nope, not required.
But terminology could be improved
s
@mbonnin lets fix those issues!
kodee loving 1
m
I was going to ask πŸ€— Are you sending the feedback to the team? Or do you want me to write this feedback somewhere?
s
If you can create a GH, or YT, issue that would be amazing. Otherwise I'll try to do it later but if you looked at the code and could provide a little bit of detail or rationale that'd be super helpful to help me prioritise it with the team kodee loving
πŸ‘ 1
j
Hmm when start server getting:
Copy code
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See <https://www.slf4j.org/codes.html#noProviders> for further details.
Exception in thread "main" java.lang.IllegalArgumentException: Neither port nor sslPort specified. Use command line options -port/-sslPort or configure connectors in application.conf
        at io.ktor.server.engine.CommandLineKt.CommandLineConfig(CommandLine.kt:74)
        at io.ktor.server.netty.EngineMain.createServer(EngineMain.kt:39)
        at io.ktor.server.netty.EngineMain.main(EngineMain.kt:24)
        at se.coody.server.ApplicationKt.main(Application.kt:13)
Not sure why I event need slf4j but ok πŸ˜„
m
@Joel Denke that's the north star for logging in JVM backend land, there's no escaping it
j
Haha ok, but why does it crash for not setup it, why isnt it just no op by default? πŸ˜„
m
Good question!
It's no-op by default actually, just with a warning
Looks like the default in ktor is to ask you to make the choices. Which is good. Every choice comes with tradeoffs. The drawback of course is that it's harder for beginners like us
Might be interesting to have
ktor
"starters", like there is in spring boot
s
πŸ‘€ 1
πŸŽ‰ 1
j
Ah wait the crash is not that sorry, its me missing out using embeddedServer, looked on bad sample πŸ˜„
☝️ 1
Ah seems I also missed adding the classic resources folder with config files as well, ah gad I hate JVM sometimes πŸ˜„
In start ktor also find out:
Copy code
application {
    mainClass = "io.ktor.server.netty.EngineMain"
}
Which is nice, to streamline all config files intro one place, neat.
m
start.ktor.io
That's useful but no substitute for a "battery-included" single maven coordinate IMO. Something like for "I want a simple JVM server that logs to stdout" (basically
ktor-starter-jvm
=
ktor-server-netty
+
slf4j-simple
)
I would use that in a bunch of small repos where I don't necessarily want to start from scratch
j
Yeah ideally want just ktor-bareminimum-graphql-with-gcp-deploy πŸ˜„
graphql 1
m
Yes, I want that as well πŸ™‚
ktor-starter-jvm
I would also probably include
cors
in there. maybe
content-negociation
, but not 100% sure about that one)
j
Often I dont care about the session and all those content negoation stuff as I want to add that later on, and start running basic basic locally on my machine only and doing pure hello world whatever endpoints in it. Maybe some basic auth stuff like CORS good add though πŸ˜„
βž• 1
Yeah at moment I added:
Copy code
private fun Application.configure() {
    install(ContentNegotiation) { json() }
    install(DefaultHeaders)
    install(CORS) {
        // TODO: replace by correct domain(s) in production.
        allowHost("localhost:8000")

        allowMethod(HttpMethod.Get)
        allowMethod(<http://HttpMethod.Post|HttpMethod.Post>)
        allowMethod(HttpMethod.Options)

        allowHeader(HttpHeaders.ContentType)
        allowHeader(HttpHeaders.Accept)
        allowHeader(HttpHeaders.Origin)
        allowHeader(HttpHeaders.Referrer)
        allowHeader("SESSION")
        exposeHeader("SESSION")
    }
}
πŸ˜„
But yeah something on my TODO, once AI get good enough, doing some kind of AI template being able to understand my domain scope. Like create new Android project with Compose and latest recommended libs and convention plugins, or create new server project with Kotlin with GCP setup and deployment. Like a template generator from AI, so I dont need to rely on someone setup this earlier πŸ˜„ But also once get there AI getting close replace everyone, so maybe best not ...
Ah finally having my own Hello world server locally now πŸ˜„
πŸŽ‰ 1
The only thing now I dont unserstand how to use this:
Copy code
externalRegistry = DockerImageRegistry.googleContainerRegistry(
            projectName = provider { "name" },
            appName = provider { project.name },
            username = providers.environmentVariable("GCLOUD_REGISTRY_USERNAME"),
            password = providers.environmentVariable("GCLOUD_REGISTRY_PASSWORD"),
        )
Not sure what USERNAME and PASSWORD to use, usually not the way of using GCP so to say, often OAuth token?
I think can do like:
Copy code
export GCLOUD_REGISTRY_USERNAME="oauth2accesstoken"
export GCLOUD_REGISTRY_PASSWORD=$(gcloud auth print-access-token)
For the easy local way of deploy. Ofc not the way of doing it longterm, then will probbly do Github Actions with Federated Identity and such.
πŸ’― 1
m
side note Issues from this discussion: β€’ https://youtrack.jetbrains.com/issue/KTOR-8746/Ktor-Gradle-plugin-consider-renaming-docker-to-jib β€’ https://youtrack.jetbrains.com/issue/KTOR-8745/Ktor-Gradle-plugin-support-configuration-cache /side note (I was wrong about the docker requirement for publishing to the Google Cloud registry, I got hit by the terminology)
πŸŽ‰ 1
kodee loving 1
Not sure what USERNAME and PASSWORD to use
I'm on the same boat. This is the part I always struggle with
j
Haha yeah I struggle with this all the time with all projects, even mobile apps. Killing me Google still using like google-play-services.json and servers on jvm using resources yaml and xml files and stuff like username and password. Havent we got further in development. Why cant stuff be like gcloud auth login and then it just works automatic if using Google COntainer automatic read from GOOGLE_APPLICATION_CREDENTIAL environment variable or something standard. And ofc using federated identity most of the time when make sense. And the GCP billing is killing me, I dont understand how to explain where I am working to non tech people add billing account to a new project I created xD
πŸ’― 1
m
If you look at the AndroidMakers source code, I use
"_json_key"
and a service account json
(which is not great from a security perspective I guess)
I remember having to "guess" that
_json_key
stuff by reverse engineering the GCloud Java SDKs or something like this
j
Really they using _json_key as username? πŸ˜„ In my case just want to use my own google auth token locally for testing. Ofc in external places would use service account json, but not really need that when logged in locally, can then use short lived token πŸ˜„
m
It's nowhere in the docs
j
Yeah why cant it be like an actual api selecting if want federated identity, service account or token, or well just automatic pick it correctly πŸ˜„
Also Google never explain which of their API support what, some Firebase stuff still not supporting Federated Identity ...
s
It uses the JIB username & password, https://github.com/GoogleContainerTools/jib/tree/master/jib-gradle-plugin#using-specific-credentials
Copy code
auth {
      username = 'oauth2accesstoken'
      password = 'gcloud auth print-access-token'.execute().text.trim()
    }
πŸ™ 2
j
Copy code
DockerImageRegistry.googleContainerRegistry(
            projectName = provider { "API" },
            appName = provider { project.name },
            username = provider { "oauth2accesstoken" },
            password = provider {
                val byteOut = ByteArrayOutputStream()
                project.providers.exec {
                    commandLine("gcloud", "auth", "print-access-token")
                    standardOutput = byteOut
                }
                String(byteOut.toByteArray()).trim()
            }
        )
So something like this then? πŸ™‚
m
project.providers.exec {}
sounds like you had a GCP problem and now you have a Gradle problem πŸ˜…
πŸ˜‚ 1
j
Yeah when running myModule:publishImage I get:
Copy code
* What went wrong:
Execution failed for task ':server:jib'.
> Could not create provider for value source ProcessOutputValueSource.
Probably related 😧
m
Yea, providers and configuration cache and lazyness is hell In those cases I do not hesitate to create my own
JavaExec
tasks nowadays. It's more code but at least it's somewhat consistent
j
Yeah created this util but seems not working well with providers ...
Copy code
fun String.runCommand(currentWorkingDir: File = file("./")): String {
    val byteOut = ByteArrayOutputStream()
    project.providers.exec {
        workingDir = currentWorkingDir
        commandLine = this@runCommand.split("\\s".toRegex())
        standardOutput = byteOut
    }
    return String(byteOut.toByteArray()).trim()
}
m
Or maybe even custom task:
Copy code
abstract class GetAccessToken: DefaultTask() {
  @get:OutputFile
  val token: RegularFileProperty

  @TaskAction
  fun taskAction() {
    ProcessBuilder()
      . // do the regular Java stuff to start a process and retrieve stdout here
  }
}
Then you register it and
Copy code
password = getAccessTokenTaskProvider.flatMap { it.token }
It's verbose as hell but at least you don't have to debug the laziness stuff
And you have 35 years of Java documentation and samples on how to use
ProcessBuilder
j
Easier just run the command manually in terminal and copy paste it each time I want to deploy πŸ˜‚
gradle intensifies 1
m
Might very well be πŸ˜‚
j
Easier just do like ./gradlew -Ppassword='gcloud auth print-token' πŸ˜„
☝️ 2
m
Right, make your shell do the
ProcessBuilder
πŸ‘
j
Yeah for now it works, just want to stop drawing the rest of the owl all the time, I want to spend time on actually coding my server so I can replace Firebase devlive KMP library.
πŸ¦‰ 1
Copy code
Execution failed for task ':server:jib'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Build image failed, perhaps you should make sure your credentials for '<http://gcr.io/project/server|gcr.io/project/server>' are set up correctly. See <https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#what-should-i-do-when-the-registry-responds-with-unauthorized> for help
Now I seems to get here at least, I guess i just need to enable some GCP stuff first πŸ˜„
Ah sweet got up running apollo execution now, wow amazing easy and integrated with automatic sandbox and everything, wow πŸ™‚
Copy code
fun main(args: Array<String>) {
    embeddedServer(Netty, port = System.getenv("POST")?.toIntOrNull() ?: 8080) {
        module()
    }.start(wait = true)
}

fun Application.module() {
    configure()
    apolloModule(ServiceExecutableSchemaBuilder().build())
    apolloSandboxModule()
}
That was finally one part being easy and setup sample doc was very good πŸ™‚
❀️ 1
m
Glad to hear it!
j
So I guess now in theory when I am able to auth myself to Firebase GCP project with FIrebase Admin SDK, I can start doing actual work with Firebase Auth and Firebase FireStore. Yeah its better doing it directly in GCP Store probably etc, but Firebase is free πŸ˜„
Best of all now, I can sync Kotlin code in my client and server in one monorepo, so much easier to develop it, instead of some external backend automagic project I need to guess how their endpoints behave, now I can see everything πŸ˜„
πŸ’― 1
I must say Ktor has grown on me a lot last years. Earlier almost always used stuff like OkHttp and NodeJs, but Ktor feels much easier to work with for both simple and complex projecs πŸ™‚
m
Didn't mention it yet but now that I can think about it, the KotlinConf app repo is another very good reference for a full stack project. The backend is included there as well
Turns out it also uses the ktor Gradle plugin but seems to bypass it to configure
jib
directly
πŸ‘€ 1
No trace of deployment code though, this must be handled out of band
j
Ah nice to see some more repos doing similar πŸ™‚ Often samples turns out to be outdated. I experienced my Compose multiplatform project to get outdated in like 1 month or so. Had to do major refactoring. But its good for learning all of this.
Yeah would love to see proper documentation of GCP and Gcloud CLI, seems to be trial and error with everything. Really insane amount of stuff can do which is hidden from documentation surface.
πŸ‘€ 1
πŸ‘ 1
s
> Turns out it also uses the ktor Gradle plugin but seems to bypass it to configure
jib
directly That might be out-of-date, and JB host KotlinConf-app on their own infra afaik (or through their own infra).
πŸ‘ 1
j
Also one thing really bother me a lot, that sample repos hardcoding maven artifact names and versions πŸ˜›
s
Really insane amount of stuff can do which is hidden from documentation surface.
Common problem in OSS though... 😞
πŸ’― 2
j
Copy code
> Task :server:run
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See <https://www.slf4j.org/codes.html#noProviders> for further details.
<===========--> 85% EXECUTING [11m 12s]
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> :server:run
> IDLE
Also this quite confusing, it says executing in Gradle but it actually running and then it doesnt print localhost:8080 up πŸ˜„
m
This is because you don't have any SLF4J provider
If you add
org.slf4j:slf4j-simple:2.0.17
to your classpath, it will show you "listening on ..."
j
Yeah that I understand, but it says nothing and executing still in process even though its not.
m
I think it is?
I mean it is executing
Even if not showing the logs
j
Copy code
<===========--> 85% EXECUTING [13m 19s]
This counting up all the time, which is weird πŸ˜„
Normally it should say only IDLE and print logs if logger added πŸ˜„
m
From the point of view of Gradle, it's still executing a task that never finishes
I agree this is a bit weird but this one TBH I don't blame ktor or Gradle on it, this is just how things work
πŸ‘ 1
Gradle could introduce a concept of "background" task but then you need a way to collect them, etc... It becomes super complicated
What I really want actually is an IJ run configuration to run things outside Gradle
Similar to what AGP is doing with "run Android app". The app is not running inside Gradle, it's a "Gradle aware make"
πŸ€” 1
πŸ’― 1
"Gradle aware make" run configuration would have 2 stages: 1. build the jar with Gradle 2. once this is done, launch the server jar in a separate JVM
This is an issue for things like compose Desktop as well
Right now, it runs the compose app inside a Gradle tasks. That's quite bad in a monorepo where you also have your server. It means: 1. you run concurrent Gradle builds, which is fundamentally unsafe 2. even worse, those concurrent builds will spawn multiple daemons allocating GB or RAM
πŸ˜ͺ 1
j
As long as I dont need to run Docker locally on computer I am happy πŸ˜„
same 1
Deploy iOS apps is even worse than desktop btw πŸ˜„
Or any "antivirus AI" for that matter impacting performance of compile and run Gradle πŸ˜‚
Follow up question, is there a recommended way once I succesfully can use JIB to publish container to GCP registry, to actually deploy it as well?
m
Cloud Run is what you want
You can create a service in the gcloud console
j
Yeah but no part of JIB or Ktor plugin to deploy this? I am using Cloud Run in Console but its very tricky to understand πŸ˜„
Want to automate it, so can deploy new versions.
But it still requires a manual step to create the service
It's a bit of a chicken and egg problem really
j
Haha ok, yeah I thought I did the deploy but only uploaded a container to GCP not actually using it πŸ˜‚
m
yup
This hardcodes the image name
j
But yeah I tested UX in console, it seems uploaded but once visit the url nothing happening, just permissions dednied.
m
Not too proud of it but I haven't found a nicer way to automate everything.
j
I expect like ./gradlew publishImageAndDeployNewRevision or something everything in one chunk, but I guess can do manual task to depend on publishImage and then actually deploy it.
j
Its harder to deploy backend than to deploy new mobile app it feels like πŸ˜„
Yeah but then need to configure Github Actions as well πŸ˜„
Copy code
tasks.register<Exec>("deployToCloudRun") {
        group = "cloud run"
        description = "Deploys the application to Google Cloud Run."
    
        dependsOn(tasks.named("jib")) 
    
        executable = "gcloud"
        // executable = "/usr/local/google-cloud-sdk/bin/gcloud"
    
        args = listOf(
            "run", "deploy", "my-api-service",
            "--image", "<http://gcr.io/project/${project.name}:${project.version}|gcr.io/project/${project.name}:${project.version}>", // Image frΓ₯n Jib
            "--project", "project",    
            "--region", "europe-west1",       
            "--platform", "managed",
            "--quiet",                        
            "--allow-unauthenticated"         
           
        )
    }
Looks like bare minimum is just adding this into my server module build gradle file πŸ˜›
I had some invalid permission setup caused issues earlier, but after solving that plus some IAM stuff to be able publish public, I finally able to publish container + deploy it on CLoud Run with GraphQL. Can ofc improve the gradle setup later on, but just glad its working automatic now πŸ™‚ Thanks for all input! Now focusing on communicate from my mobile app to my server instead. Thanks everyone!
m
Congrat,πŸ‘πŸ‘
πŸ‘ 1
j
Thanks πŸ™‚ Let me know if want me to share more of my setup of something, or if have any ideas ofc how improve things. Especially I wished that Ktor plugin could handle both publish container and deploy it in same plugin, and yes ofc decouple them in two different tasks, quite intense amount of things to think about πŸ˜„ Sure Gcloud CLI is nice, but its very complex what parameters should use and when, its then most oten nice having default values in a plugin with best practice values for most users. But yeah not that amount of work write my own things.
πŸ‘€ 1
Ideally I would create Gradle plugin for this, to handle not only Ktor though, but also other servers like Spring or such πŸ˜„ Sadly I dont have time write all of these plugins I want to write OSS. I have a LOOOOONG list if OSS things I want to do, like OSS banking or OSS CMS multiplatform and cross programming language, feels waste of time re-invent CMS and bank systems over the world over and over again in bad way. So annoying no one can build a proper headless CMS working on all platforms πŸ˜„ Where can have CMS adapters for things like textcopy so can use third party tooling inside CMS system, and not silver bullet. But yeah I am glad there is a Ktor plugin existing, saved me a lot of time not write JIB things manually.
Also other ideas for Ktor is provide Github Action plugin, to deploy with best practices of Federated Identity + deploy process in CI pipelines. I love how Gradle and now also Google and Jetbrains adding Github Action plugins for their things.
As most probably make most sense to deploy in CI workflow and most probably not locally. But can ofc combine gradle task and stuff like GHA πŸ™‚
m
Did you get federated identity working in the end?
j
As I am only run locally at the moment there is nothing to apply with federated identity as running locally with my own Google account. But once I am adding CI pipeline for this, which I probably will soon, then I will ofc use Github Actions with federated identity thingy, which is quite easy. I have used it in the past, so not complex to add πŸ™‚ If I recall correctly it will get like GOOGLE_APPLICATION_CREDENTIALS or something like that environment variable autoamtic from that federated identity which generate from enterprise scope thing need to setup on Github Secrets or something like that, dont remember πŸ˜›
πŸ‘ 1
m
Gcloud CLI is nice, but its very complex what parameters should use and when, its then most oten nice having default values in a plugin with best practice values for most users.
I'm not a fan of this trend to ship CLI binaries instead of libs. It makes everything type unsafe, requires parsing output, etc... Plus we have to find another way to keep the versions up to date, etc.... </rant>
j
Yeah totally agree, but there is so many CLI fans out there. Most sys ops and such prefer terminals and such. So there is use case for that. But I wish we could decice ISO standards for developers, to have some kind of concensus of tooling and not reinvent wheel in Github Actions/Jenkins/CircleCI etc vs CLI and Gradle Plugins and all other programming language ecosystems πŸ˜„ Not all languages has type safety either, so its eh funny. But at least would be nice in Kotlin world having 1-3 thing and not 100+ different things doing same thing πŸ˜„
m
One day everybody will write typescript and yaml 🧌
j
Hopefully Typescript die soon, I am so tired of it πŸ˜„ Funny write Gradle with Yaml btw with Amper πŸ˜„